Skip to content

feat(litellm): emit streaming latency span attrs - #39

Merged
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
feat/litellm-streaming-latency-attrs
Jun 19, 2026
Merged

feat(litellm): emit streaming latency span attrs#39
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
feat/litellm-streaming-latency-attrs

Conversation

@sayonfortify

Copy link
Copy Markdown

Adds FortifyRoot streaming-latency span attributes to the LiteLLM streaming safety wrapper.

What changed:

  • sync and async LiteLLM streaming wrappers set:
    • fortifyroot.llm.streaming.time_to_first_token_ms
    • fortifyroot.llm.streaming.time_to_generate_ms
  • first token is detected only from output-bearing chunks (delta.content, message.content, text/content); role-only or empty chunks do not start timing.
  • empty streams leave both attrs unset so downstream RDS fields remain NULL, not fake 0 ms.
  • preserves existing streaming safety masking, accumulation, status, context detach, and span-end lifecycle.

Tests:

  • conda py3.12: python -m pytest packages/opentelemetry-instrumentation-litellm/tests/test_safety_hooks.py -q → 31 passed
  • conda py3.12: python -m ruff check touched files → pass
  • python3 -m py_compile touched files → pass
  • git diff --check → clean

Vendoring note:
Per internal-sop/openllmetry/VENDORING_SOP.md, ocelle-py vendoring is intentionally not done in this PR. After this merges, cut the next fr-v0.52.6.N tag on the fork and run ocelle-py/scripts/vendor.sh against that tag.

@sayonfortify
sayonfortify merged commit f1fbd88 into fr-v0.52.6.x Jun 19, 2026
10 checks passed
@sayonfortify
sayonfortify deleted the feat/litellm-streaming-latency-attrs branch June 19, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant